Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CH-384] add filecache and link jemalloc in the same order as CH #386

Open
wants to merge 749 commits into
base: clickhouse_backend
Choose a base branch
from

Conversation

binmahone
Copy link

Changelog category (leave one):

  • New Feature
  • Improvement
  • Bug Fix (user-visible misbehavior in official stable or prestable release)
  • Performance Improvement
  • Backward Incompatible Change
  • Build/Testing/Packaging Improvement
  • Documentation (changelog entry is not required)
  • Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

...

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

liuneng1994 and others added 30 commits April 1, 2022 10:39
Backport ClickHouse#35755 to 22.3: Fix bug in conversion from custom types to string
Backport ClickHouse#35770 to 22.3: Fix enable LLVM for JIT compilation in CMake
…ash_v2

One more try to resurrect build hash
taiyang-li and others added 18 commits March 6, 2023 16:40
Upgrade substrait proto by velox

Co-authored-by: shuai.li <[email protected]>
…House#354)

ShuffleSplitter improvement: support multiple subdirs
Support full join with join condition

Co-authored-by: shuai.li <[email protected]>
@kyligence-git
Copy link
Collaborator

Can one of the admins verify this patch?

String region_name;
const char * amazon_suffix = ".amazonaws.com";
const char * amazon_prefix = "https://s3.";
if (endpoint.find(amazon_suffix) != std::string::npos)
Copy link
Collaborator

@liuneng1994 liuneng1994 Mar 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个逻辑可能不适用于aws cn

@@ -165,7 +234,7 @@ void registerReadBufferBuildes(ReadBufferBuilderFactory & factory)
factory.registerBuilder("hdfs", [](DB::ContextPtr context_) { return std::make_shared<HDFSFileReadBufferBuilder>(context_); });

#if USE_AWS_S3
factory.registerBuilder("s3", [](DB::ContextPtr context_) { return std::make_shared<S3FileReadBufferBuilder>(context_); });
factory.registerBuilder("s3a", [](DB::ContextPtr context_) { return std::make_shared<S3FileReadBufferBuilder>(context_); });
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里是不是可以同时支持s3和s3a

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可以 但是因为spark上层好像只认s3a 不认s3 ,所以这里加了以后,相当于没有测试到

@@ -165,7 +234,7 @@ void registerReadBufferBuildes(ReadBufferBuilderFactory & factory)
factory.registerBuilder("hdfs", [](DB::ContextPtr context_) { return std::make_shared<HDFSFileReadBufferBuilder>(context_); });

#if USE_AWS_S3
factory.registerBuilder("s3", [](DB::ContextPtr context_) { return std::make_shared<S3FileReadBufferBuilder>(context_); });
factory.registerBuilder("s3a", [](DB::ContextPtr context_) { return std::make_shared<S3FileReadBufferBuilder>(context_); });
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里是不是可以同时支持s3和s3a

…AD, and set ENABLE_JEMALLLOC=OFF,and JEMALLOC_ATTR(constructor(101)) in jemallo.c)
…le PRELOAD, and set ENABLE_JEMALLLOC=OFF,and JEMALLOC_ATTR(constructor(101)) in jemallo.c)"

This reverts commit f242fcc.
@binmahone binmahone force-pushed the 230327_support_s3_and_cache branch from d5905eb to 21d5919 Compare April 12, 2023 07:03
@lwz9103 lwz9103 force-pushed the clickhouse_backend branch 2 times, most recently from dc60d55 to 8066113 Compare May 26, 2023 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.